1. Field of the Invention
The present invention relates to a signal processing apparatus, image sensing apparatus, image sensing system, and signal processing method.
2. Description of the Related Art
In a conventional image sensing apparatus such as a digital camera or a digital video camera, a CCD image sensor, a CMOS image sensor, or the like is used as an image sensor.
An image sensor IS shown in FIG. 13 is provided with a pixel array PA in which a plurality of pixels are arranged in a direction along a row and a direction along a column. Each of pixels P1 to P5 in the pixel array PA includes a photoelectric conversion unit PD, a color filter CF, and a microlens ML. In each of the pixels P1 to P5, an open area OA between the color filter CF and microlens ML and the photoelectric conversion unit PD is defined by wiring layers WL. The photoelectric conversion units PD of adjacent pixels are electrically separated from each other by an element isolation region IR. The photoelectric conversion units PD and the element isolation regions IR are disposed within a semiconductor substrate SB.
A case is considered in which, in FIG. 13, the color filter CF of the pixel P1 transmits light of a first color, and the color filter CF of the pixel P2 transmits light of a second color. There may be an instance where first color light IL1′ among diagonally incident light IL1 on the pixel P1 passes through the open area OA of the pixel P1, and then a portion IL1″ of the first color light IL1′ is transmitted through the element isolation region IR and arrives at the photoelectric conversion unit PD of the adjacent pixel P2. In this case, although the photoelectric conversion unit PD of the pixel P2 should properly receive the light of the second color, the photoelectric conversion unit PD further receives the first color light IL1″ mixed in from the adjacent pixel, and generates a signal corresponding to the first color light IL1″. That is, a so-called color mixture phenomenon may occur in which the signals of adjacent pixels interfere with each other.
The ease with which this color mixture occurs differs according to the F value (stop aperture diameter) of a shooting lens in the image sensing apparatus, as shown in FIGS. 14A and 14B. Compared to a case where the stop aperture diameter is small as shown in FIG. 14B, in a case where the stop aperture diameter is large as shown in FIG. 14A, diagonal incident light more easily mixes into an adjacent pixel. That is, as the F value of the shooting lens becomes smaller, the stop aperture diameter becomes larger, and there is a tendency for the amount of diagonal incident light that mixes into an adjacent pixel to increase.
Also, the ease with which color mixture occurs differs according to an exit pupil distance (distance from the image plane to the exit pupil position of the shooting lens) of the shooting lens in the image sensing apparatus, as shown in FIGS. 15A and 15B. Compared to a case where the exit pupil distance is long as shown in FIG. 15A, in a case where the exit pupil distance is short as shown in FIG. 15B, diagonal incident light more easily mixes into an adjacent pixel. That is, as the exit pupil distance becomes shorter, there is a tendency for the amount of diagonal incident light that mixes into an adjacent pixel to increase.
As shown in FIG. 16, the photoelectric conversion unit PD of the pixel P2 receives second color light IL2′ that has been transmitted through the color filter CF of the pixel P2 among the incident light IL2 on the pixel P2, and generates charges (signal) according to the received second color light IL2′. Furthermore, the photoelectric conversion unit PD of the pixel P2, which has received the first color light IL1″ from the adjacent pixel P1 for the reasons shown in FIGS. 14A, 14B, 15A, and 15B, generates charges (signal) according to the first color light IL1″, as shown in FIG. 16. Thus, the photoelectric conversion unit PD of the pixel P2 generates a signal according to the first color light IL1″ in addition to a signal according to the second color light IL2′, and thereby color mixture occurs.
Also, the ease with which color mixture occurs differs according to the color of light incident on the photoelectric conversion unit PD, as shown in FIGS. 16 and 17. This is because the depth from the surface of the semiconductor substrate SB at the position where light is converted to charges in the photoelectric conversion unit PD differs according to the wavelength of the light. That is, this is because, in comparison to light having a short wavelength, light having a long wavelength is photoelectrically converted at a deeper position in the photoelectric conversion unit PD.
Here, the color filter CF of the pixel P1 shown in FIG. 16 transmits red (R) light, the color filter CF of the pixel P2 transmits green (G) light, and the color filter CF of a pixel P6 shown in FIG. 17 transmits blue (B) light.
As shown in FIG. 16, light that has passed through the red (R) light color filter CF, in comparison to light that has passed through the color filters CF of the other colors (G, B), is photoelectrically converted at a deeper position in the photoelectric conversion unit PD. Therefore, the red (R) light IL1′ passes through the photoelectric conversion unit PD of the pixel P1 where that light should be incident, and a portion IL1″ of that light easily becomes incident on the photoelectric conversion unit PD of the adjacent pixel P2. The light IL1″ that is incident on the photoelectric conversion unit PD of the adjacent pixel P2 is photoelectrically converted there, so without producing charges (signal) of the pixel P1 where the light should be incident, a mixed color component for the signal of the adjacent pixel P2 is generated.
On the other hand, as shown in FIG. 17, light IL6′ that has passed through the blue (B) color filter CF, in comparison to light IL1′ and IL2′ that has passed through the color filters CF of the other colors (R, G), is photoelectrically converted at a shallower position in the photoelectric conversion unit PD. Therefore, even if a light ray IL6 is diagonally incident on the pixel P6, there is a tendency for the light ray to be photoelectrically converted in the photoelectric conversion unit PD of the pixel P6 prior to arriving at the photoelectric conversion unit PD of an adjacent pixel P7. That is, because it is unlikely that the blue (B) light IL6′ will pass through the photoelectric conversion unit PD of the pixel P6 where that light should be incident and arrive at the adjacent pixel P7, it is unlikely that a mixed color component for the signal of the adjacent pixel P7 will be generated.
Also, as shown in FIG. 18, at a deep position in the semiconductor substrate SB, between the photoelectric conversion units PD of adjacent pixels, electrical separation by the element isolation region IR is inadequate. Therefore, charges (signal) that are stored at a deep position in the photoelectric conversion unit PD of the pixel P1 are dispersed and mixed into the photoelectric conversion unit PD of the adjacent pixels P2 and P4 at a deep position in the semiconductor substrate SB. This crosstalk within the semiconductor substrate SB also causes color mixture.
The ease with which color mixture due to this crosstalk occurs differs according to the color light that is incident on the photoelectric conversion unit PD, as shown in FIGS. 19 and 20.
As shown in FIG. 19, light IL1′ that has passed through the red (R) color filter CF, in comparison to light that has passed through the color filters CF of the other colors (G, B), is photoelectrically converted and stored at a deeper position in the photoelectric conversion unit PD. Therefore, charges (signal) stored in the photoelectric conversion unit PD of the pixel P1 according to the red (R) light IL1′ easily passes, at a deep position in the semiconductor substrate SB, through the area deeper than the element isolation region IR and is dispersed in the photoelectric conversion units PD of the adjacent pixels P2 and P4. In the area deeper than the element isolation region IR, it is conceivable that electrical separation is inadequate between the photoelectric conversion units of adjacent pixels. Thus, charges (signal) dispersed in the photoelectric conversion units PD of the adjacent pixels P2 and P4 easily generate mixed color components for the signal of the adjacent pixels P2 and P4, without becoming the charges (signal) of the pixel P1 where the dispersed charges (signal) should be stored.
On the other hand, as shown in FIG. 20, light IL6′ that has passed through the blue (B) color filter CF, in comparison to light that has passed through the color filters CF of the other colors (R, G), is photoelectrically converted at a shallower position in the photoelectric conversion unit PD. Therefore, charges (signal) stored in the photoelectric conversion unit PD of the pixel P6 according to the blue (B) light IL6′ is blocked, at a shallow position in the semiconductor substrate SB, by the element isolation region IR, and is unlikely to be dispersed into the photoelectric conversion unit PD of an adjacent pixel P8. The charges (signal) stored in the photoelectric conversion unit PD of the pixel P6 according to the blue (B) light IL6′ are unlikely to generate mixed color components for the signal of the adjacent pixel P8.
Due to color mixture that occurs in this manner, the image signal that is output from the image sensor deteriorates, and thereby color reproducibility deteriorates.
Japanese Patent Laid-Open No. 2004-135206 describes that, in a CCD image sensing element having a color filter array according to a Bayer array, color mixture correction subtracts, from the signal of a designated color pixel, a fixed ratio calculated from the signal of the designated color pixel and the signal of an adjacent pixel of a color other than the designated color.
In this correction processing, it is assumed that color mixture occurs, relative to a pixel of interest, isotropically from a plurality of surrounding pixels that are adjacent to that pixel of interest, i.e., that a signal component is mixed in at the same ratio from a plurality of surrounding pixels. Under this assumption, a signal component of a fixed ratio is isotropically subtracted.
On the other hand, Japanese Patent Laid-Open No. 2007-142697 describes that, in an actual solid image sensing element, the light receiving face of a photoelectric conversion unit is disposed at an offset position within a pixel, depending on the wiring pattern and the layout of electrodes within the pixel or in the vicinity of the pixel. As a result, the physical center of the pixel and the optical center of the pixel do not match, and thereby color mixture from surrounding pixels relative to the pixel of interest can be made to occur with directionality.
To address this problem, Japanese Patent Laid-Open No. 2007-142697 proposes changing, independent of each other, correction parameters Ka, Kb, Kc, and Kd for correcting color mixture from surrounding pixels respectively at the upper left, upper right, lower left, and lower right. Thus, according to Japanese Patent Laid-Open No. 2007-142697, it is possible to realize correction processing of color mixture that is made to have directionality according to the amount of color mixture from the surrounding pixels.
In Japanese Patent Laid-Open No. 2007-142697, a correction circuit for performing color mixture correction processing receives a control signal of directionality selection supplied from outside via a communications I/F, and changes the correction parameters Ka, Kb, Kc, and Kd independent of each other according to the received directionality selection control signal. Specifically, when the directionality selection control signal that has been supplied from outside via the communications I/F is 0, Ka=Kb=K1, and Kc=Kd=K2 are set. When the directionality selection control signal is 1, Ka=Kc=K1, and Kb=Kd=K2 are set, and when the directionality selection control signal is 2, Ka=Kd=K1, and Kb=Kc=K2 are set. In the correction circuit described in Japanese Patent Laid-Open No. 2007-142697, the correction parameters Ka, Kb, Kc, and Kd used for the respective signals of a plurality of pixels disposed in the solid image sensing element have values that are common to each pixel. However, strictly speaking, in the actual image sensing apparatus, the angle of incident light rays on the pixels differs according to the arrangement of pixels in a sensor face. The amount of color mixture from an adjacent pixel for a pixel of interest differs according to the light ray incidence angle, i.e., the amount of color mixture in each pixel differs according to the pixel arrangement of respective pixels in the sensor face. In the case of such color mixture that occurs in a non-uniform manner in the sensor face, with the correction circuit described in Japanese Patent Laid-Open No. 2007-142697, there is a high possibility that the accuracy of color mixture correction processing will deteriorate according to the position of pixels in the sensor face (pixel array).